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Abstract 

Landauer's principle states that the erasure of infor- 
mation must be a dissipative process. In this paper, 
we carefully analyze the recording and erasure of in- 
formation on a physical memory. On the one hand, 
we show that in order to record some information, the 
memory has to be driven out of equilibrium. On the 
other hand, we derive a differential version of Lan- 
dauer's principle: We link the rate at which entropy 
is produced at every time of the erasure process to 
the rate at which information is erased. 



1 Introduction 

The fundamental role of information in thermody- 
namics is now well established. On the one hand, 
it is possible to convert information into useful work 
and use it as a "fuel" for some heat engine (TJ [2] . On 
the other hand, information itself must be recorded 
on some physical memory. Information processing is 
implemented via physical transformations operating 
on the memory and is thereby accompanied by ther- 
modynamic costs [3lil[5l|6l[illi[i|TOl[TTl|T2]. The 
paradigmatic result concerning the costs of informa- 
tion processing is Landauer's erasure principle (3^. It 
is a statement about the thermodynamics of infor- 
mation erasure. In its original formulation, it states 
that the resetting to zero of a random bit necessarily 
leads to the dissipation of A;BTlog2 of energy where 
is Boltzmann's constant and T is the temperature 



of the environment. Landauer's principle was cele- 
brated as the resolution Maxwell's demon's paradox 
[13] . However it is still a very controversial result, see 
[14| and references therein. In this paper, we care- 
fully analyze the processes of recording and erasing 
information on a physical memory in a simple but 
generic scenario and we precisely derive the thermo- 
dynamic costs of these processes. While the problem 
usually addressed in the literature is the operation 
"reset to zero" performed on isolated one-bit memo- 
ries [m [H [m [HI [H [g [6], here we propose to an- 
alyze the recording and erasure of some information 
coming from an external source. We use the frame- 
work of stochastic thermodynamics [201 [^[^[^[^ 
to model the memory and to derive the thermody- 
namics of these processes. 

We consider the following scenario: A source ran- 
domly emits a symbol ak out of N possible symbols, 
Qfjv. We wish to record which symbol ap- 
peared on some physical memory and then we wish 
to erase it. The memory should be able to be in 
at least N states: One for each of the possible out- 
comes. However, it is convenient to have one more 
state serving as the "standard" state: It is the state 
of the memory when it is empty, i.e. when nothing 
is recorded. Recording ak simply means to drive the 
memory from the standard state to state number k. 
Erasing the content of the memory means to drive it 
back to the standard state by a process that is inde- 
pendent of the symbol stored. 

In the following, we show that recording the in- 
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formation amounts to correlate the memory to the 
symbol emitted by the source. During the era- 
sure process, these correlations gradually decrease to 
zero. We show that the rate of entropy produced is 
bounded from below by the rate at which the corre- 
lations decrease all along the erasure process. This is 
our second main result, eq. (1101) . which can be viewed 
as a differential version of Landauer's principle. Our 
first main result, eq. ([S]) is a statement about a differ- 
ent kind of costs for the correlations: Although the 
recording can in principle be performed reversibly, 
the memory can only store some information if it is 
out of equilibrium. 



2 Recording information on a 
physical system 

In order to simplify the discussion, we use an over- 
damped particle subject to a random force of ther- 
mal origin as a memory. However, any system obey- 
ing the laws of stochastic thermodynamics would do. 
The particle is also subjected to a force F{x, A) = 
^dxV{x, A) derived from a conservative potential 
V{x,X). The potential is time-dependent through 
the external control parameter A that can be varied 
in order to control the state of the memory. The posi- 
tion of the particle evolves according to the Langevin 
equation: 

i = fiF{x,x)+at), (1) 

where /i is the mobility of the particle and ^{t) 
is a Gaussian white noise with {£,{t)) — and 
(^(t)^(t')) = 2DS{t - t'), D being the diffusion con- 
stant. When the medium is in equilibrium at tem- 
perature T, the diffusion constant and the mobility 
are related by the Einstein relation D = k-QTji. The 
probability density p{x, t) to find the particle at po- 
sition X at time t evolves according to the Fokker- 
Planck equation: 

dtp{x, t) = -dx {fiF{x, X)p{x, t) - Ddxp{x, t j) , (2) 

Because of thermal fluctuations, the position of the 
particle is random and we cannot control it. However, 
through a suitable choice of the time dependence of 



the control parameter X{t) (the protocol), we can con- 
trol the distribution p{x, t) describing the position of 
the particle. Hence, we will use such distributions to 
encode the different symbols a^. 

Let {(/)fe},0<fc<iVbeA^-|-l distributions over 
the position of the particle: 4>o{x) is the standard 
state and for 1 < fc < A^, (t>k{x) encodes the sym- 
bol afc. For the information to be unambiguously 
recorded, we need the states encoding different sym- 
bols to be perfectly distinguishable. This means that 
the corresponding distributions should not overlap: 
For each position x there should be only one k such 
that 4>kix) is non zero. In fact, the information is 
stored in the position of the particle. If for a given 
position X, (j)k{x) > and (j)k'{x) > 0, then observing 
the particle at x we cannot decide whether or ak' 
is stored. 

The recording process is the following. Assume 
that the symbol emitted by the source is ak- Ini- 
tially, the memory is in the standard state p{x,to) = 
00 (x) and the control parameter takes stored value 
A(io) — Aq. From time t = to < to i = 0, the 
control parameter is changed from its initial value 
Ao to some final value Arec according to some proto- 
col Xk{t)- The protocol depends on the symbol Uk 
that appeared and is such that the final state of the 
memory is p{x,Q) = 4>k{x). However, once the infor- 
mation is recorded, we want to be able to manipulate 
the memory without knowing the information stored. 
Hence, the final value of the control parameter, Aroc 
should be the same for all k as illustrated on figure 
[TJ As a consequence, at the end of the recording pro- 
cess, the memory is out of equilibrium. In fact we 
want to allow it to be in one out of N different states 
for a single value of A. But for each value of A, there 
is only one equilibrium state given by the Boltzmann 
distribution: 



Poq(a;, A) = exp 



V{x,X)-F,^{X) 
kjiT 



(3) 



where exp(— Fcq(A)/fcBT) = 

J cxp{~V{x, X)/kBT)dx is the partition func- 
tion and Fcq{X) the equilibrium free energy as a 
function A. 

In stochastic thermodynamics, the distance of a 
non equilibrium state p to equilibrium p^q is quan- 
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Figure 1: Recording protocol: From time i = < 
to t = 0, the control parameter A is driven from 
A(<o) — Ao to A(0) = Arec in a way that depends on 
fc, such that the final state of the memory is 0^, the 
state encoding a^, the symbol that was emitted by 
the source. 

tified by the KuUback-Leibler divergence, or relative 
entropy between p and pcq [lH |22l [26] : 

^[pllPoq] = / p(a;)log^^da;. (4) 

In fact, this quantity is linked to the free energy of the 
non equilibrium state p: F\p\—F^g^ = k-QTD\p^p^^ > 
[ini [121 [IS] ■ The relative entropy between two dis- 
tributions is non negative and it vanishes if and only 
if the two distributions are identical [27] . 

The first result of this paper is the following in- 
equality: 

J2PkD[<j)k\\Pcq]>H, (5) 

k 

where poq is the final equilibrium distribution, Pk is 
the probability that the symbol was emitted and 
H = —J2k ^k^ogPk is the Shannon entropy of the 
source. The latter quantifies our a priori uncertainty 
about the symbol emitted by the source or the aver- 
age information provided by the observation of the 
symbol emitted [37]. Inequality ([5]) above means, 
that the expected distance to equilibrium at the end 
of the recording process is greater than the average 
information provided by the emission of the symbol 
to be recorded. One can interpret H as the amount 
of information stored in the memory. Inequality ([5]) 
can then be interpreted in the following way: In or- 
der to record some information, the system serving 
as a memory has to be driven out of equilibrium and 



the distance to equilibrium has to be greater than the 
information stored. Nevertheless, the recording pro- 
cess can in principle be performed reversibly in that 
there is no finite lower bound to the entropy that has 
to be produced. 

Inequality ([5]) is very general and does not depend 
on the structure of the memory. The only assump- 
tions are: (i) The macroscopic states of the mem- 
ory are given by distributions over the micro-states, 
(ii) states of the memory encoding different symbols 
should be perfectly distinguishable, (iii) there is a 
unique equilibrium distribution. Sometimes, due to 
technical restrictions, it might not be possible to pre- 
pare the memory in non overlapping states and hence 
it would not be possible to fulfill condition (ii) . In this 
case the information cannot be completely unambigu- 
ously recorded, but nevertheless, as we will see later, 
one can still quantify the maximum amount of infor- 
mation, /max < H, that can be stored and inequality 
([5]) will still be valid replacing H by /max in the right 
hand side. We will address this point later, after hav- 
ing discussed the erasure process. 

3 Erasure: differential Lan- 
dauer's principle 

The erasure is a process bringing the memory back 
to its standard state without making use of the infor- 
mation stored. This is expressed by the fact that the 
erasure protocol does not depend on k. The initial 
value of the control parameter is Arcc and the initial 
state of the memory is the state encoding the symbol 
that was emitted, i.e. it is (j)k{x) with probability Pk- 
From time t = to time t = ti the control parameter 
is driven from Aioc back to Aq in such a way that the 
final state is the standard state 4>o{x). 

Assume that we stop the process at some interme- 
diate time t. We would like to address the follow- 
ing issues: What is the information erased until that 
time? Or alternatively, what is the information still 
contained in the memory? And what is the minimum 
amount of entropy produced until that time? 

In order to illustrate the information loss, consider 
the following situation: At time t, we see the particle 
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at position x and from this knowledge, we would like 
to infer which symbol was stored. The probability 
P{k\x; t), that the symbol ak was stored is given by 
Bayes rule: 



P{k\x;t) = 



Pk{x,t)Pk 



(6) 



where pkix,t) is the distribution of the particle's 
position at time t if ak was stored and pra{x,t) — 
X^A: PkPk{x, t) is the marginal distribution of the po- 
sition of the particle at time t. Its presence in the 
expression above ensures that the probability distri- 
bution P{k\x;t) is normalized. Concretely, pk{x,t) 
is obtained by propagating (j)k{x) with the Fokker- 
Planck equation with the erasure protocol X{t). Ini- 
tially, ai t — Q, P{k\x]Q) = or 1 depending on 
whether x belongs to the support of <j)k of not: The 
position of the particle contains the complete infor- 
mation about the symbol that was emitted. At in- 
termediate time t, the pk{x,t) might overlap and we 
will have some uncertainty about the symbol that was 
stored upon seeing the particle at position x. This 
uncertainty is quantified by the Shannon entropy of 
the probability distribution P{k\x; t): 

h,,{x,t) = - ^P(fc|x;i)logF(fc|a;;t). (7) 

k 

The total information erased until time t is the av- 
erage uncertainty about the symbol that was stored 
upon knowing the position of the particle at time t: 



Hcr{t) = / p„^{x,t)ha{x,t)dx. 



(8) 



At the beginning of the erasure process, no informa- 
tion is yet erased and -ffer(O) = 0. At the end of 
the process, knowing the position of the particle does 
not reduce our uncertainty about the symbol that 
had been emitted and i?or(ii) = H. The information 
I(t) still contained in the memory is the reduction in 
uncertainty about the symbol that was stored upon 
knowing the position of the particle at time t: 



I{t) = H - H,,{t). 



(9) 



This is just the mutual information between the po- 
sition of the particle and and the symbol originally 



recorded. It is a measure of how much information 
the position of the particle at time t can still pro- 
vide about the symbol originally stored [37|. During 
the erasure process, it decreases from _ff to as the 
information is gradually erased. 

The second main result of this paper is the follow- 
ing: The rate of entropy production is bounded from 
below by the rate of information erasure: 



5'"- > knH.r = -knl. 



(10) 



We call this result the differential Landauer's prin- 
ciple. In fact, it is a precise and general statement 
of the thermodynamic costs of information erasure. 
Integrating the equation above between times t and 
t' , with < t < t' < ti, we get a lower bound for the 
irreversible entropy production between time t and 
t': 



AS'"{t,t')>kB (lit) -lit')). 



(11) 



Hence, the minimum amount of entropy produced 
is directly linked to the loss of correlation between 
the position of the particle and the symbol initially 
stored. Setting t — and t' = ti in the equation ([TT|) 
above gives a general integral version of Landauer's 
principle: 



AS"" > kBH, 



(12) 



linking the total amount AS""^ of entropy produced 
to the amount H of information erased. 

In general, we might not be able to prepare the 
memory in non overlapping states {(f>k}- In this case, 
the information cannot be fully reliably stored. How- 
ever, we can still quantify the maximum amount of 
information that we can store. It is given by eq. 
for t = 0: /max = /(O) ^ H - iJo,.(0) < H, where 
iJor(O) > due to the overlapping. In this case, in- 
equality ([5]) still holds replacing H by /max hi the 
right hand side. Moreover, the lower bound to the 
total entropy produced during the erasure process, 
obtained by integrating inequality (|10l) . is reduced: 
AS'"''' > fee /max, implying that we do not have to 
pay for the information we were not able to record. 
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4 Proof of equations ([5]) and 

m 

We now prove our two main results, inequalities ([5]) 
and (llOp . The information contained in the memory 
at time t of the erasure process can be expressed as: 

i(t) = Y,PkD[pkmpm{t)]. (13) 

k 

Using this expression, it is easy to show that the av- 
erage distance to equilibrium satisfies: 

Y,PkD[pkmpoc,{xm ^ i{t)+D[prnmpoc,{xm- 

k 

(14) 

At time t — Q, this expression implies inequality ([5]), 
since /(O) = i?, and the second term in the right hand 
side is non negative at all times. Actually, eq. (|T4)) is 
a generalization of inequality ([5]). It relates the av- 
erage distance to equilibrium to the information still 
contained in the memory at any time of the erasure 
process. 

At time t of the erasure process, if ak was stored, 
the state of the memory is Pk{x, t) introduced earlier. 
The thermodynamic entropy of the memory is given 
by the Shannon-Gibbs formula: 

Sk{t) ^ -ks J pk{x,t) log pk{x,t)dx. (15) 

Its variation satisfies the Clausius relation [22l [28] : 

s,it)^9M + snt), (16) 

where Qk{t) = J V{x, X{t))dtPk{x,t)dx is the heat 
fiux to the particle at time t and S^i^'^{t) > is the 
rate at which entropy is irreversibly produced. 

Using eq. (fT3|) . we get the following relation be- 
tween the information I{t) and the expected entropy 
S{t) ^J2k PkSk{t) of the memory: 

5(t)-^,„(t)-fcB/(t), (17) 

where S'm(0 — / Pm(a;, i) logpin(2:, i)da; is the 
entropy of the marginal distribution p-ai{x,t). Since 
the Fokker-Planck eq. ([2]) is linear and since the 



marginal distribution is a linear combination of so- 
lutions of this equation, it satisfies this equation as 
well. As a consequence, the variations of S^{t) also 
satisfy some Clausius relation similar to eq. (jl6p : 

S.M = ^ + S^{tl (18) 

where Qm{t) — J V{x, X{t))dtPni{x,t)dx and 

Noting that Qm(0 = J2k PkQk{t) and inserting 
equations and into eq. pT)) yields for the av- 
erage entropy production rate S^"{t) = J2k PkS"'^{t)- 

^"■■■(0 = s^:\t) - ksiit). (19) 

This proves eq. ^ since Sl^' {t) > 0. 

We used an over-damped Brownian particle as a 
memory in order to present our results clearly. How- 
ever, one could use other types of systems obeying 
the laws of stochastic thermodynamics as a mem- 
ory. In particular, systems with discrete phase space 
evolving according to master equations such as the 
systems considered in [29l [23] yield the same results, 
where it is essential that they fulfill the Clausius rela- 
tion, eq. (jl6p . In fact in the following we will present 
the recording and erasure of the outcome of a bi- 
nary random variable on a memory with two discrete 
micro-states. 

5 Example: a two states sys- 
tem as a memory 

As an example, we consider the recording and erasure 
of the result of a binary random variable. Let Y be 
a random variable with two possible outcomes. In 
analogy with the tossing of a biased coin, let "head" 
and "tail" be the two possible outcomes appearing 
respectively with probability P and 1 — P. We want 
to record the outcome of Y and then erase it. The 
amount of information we want to store is given by 
the Shannon entropy of Y, H{P) = — PlogP — (1 — 
P)log(l-P). 

The system serving as a memory is a two states 
system in contact with an equilibrium heat bath at 
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Figure 2: A very simple model for a memory: A 
system that can make thermally activated transition 
between two states "L" and "R". This system can 
be controlled through the energy difference between 
the two states. Its macroscopic state is given by the 
probability p to occupy the state "L". This system 
could represent a Brownian particle in a double well 
potential, that can jump from one well to the other 
because of thermal fluctuations. 

temperature T. This system could represent an over- 
damped Brownian particle in a double well potential 
as depicted in figure [2] In the following, we will 
again speak of a "particle" that can "jump" between 
two "wells" keeping in mind that the memory could 
be any two states system in contact with a heat bath 
that can make transitions between the two states. Let 
"L" and "R" label the two wells and let Ei^ = -E 
and Eji = +E be their respective energies. We as- 
sume that we are able to control the energy difference 
AE = Eji — _El = 2E between the two wells by ap- 




Figure 3: (a) Imnx/H as a function of E^^^^/k^T 
for P — 0.8. The result is very similar for other 
values of P. (b) Kullback-Leibler divergence D — 
P D{p\-i\\pc(i) + [l — P)D{pt\\pcq) to equilibrium at the 
end of the recording process and maximum informa- 
tion /max stored in the memory as a function of Pmax- 



plying a force field. Hence, our control parameter is 
E. 

At any time the particle might jump from one well 
to the other due to thermal fluctuations of the heat 
bath. The probability per unit time, that the particle 
jumps from the right to the left well (from the left 
to the right well) is given by Kramer's rate wr = 
r-iexp(£;//cBr) (wl = t'^ exp (-£:/A:bT)), where 
r is a time linked to the height of the potential barrier 
between the two wells. The macroscopic state of the 
memory is fully described by the probability p that 
the particle is in the left well. The latter evolves in 
time according to the following master equation: 

p = + (1 - p) . (20) 

The rates satisfy detailed balance uir/wl = 
exp(— Ai^/feB?^) and the equilibrium is given by the 
Boltzmann factor 

Peq(i?)=exp(^-:^-^), (21) 

where the equilibrium free energy F{E) is linked to 
the partition function Z{E) = c^j>{- F (E) / kBT) = 
2cosh(£'/fcB2^)- The thermodynamic entropy of the 
memory is given by S{p) = —kB{p\ogp -I- (1 — 
j')log(l — p)) and the heat received per unit time 
is given by Q = pAE = 2pE. The instantaneous 
entropy production rate is given by the Clausius re- 
lation S"' ^ S -Q/T>0. 

Now, we need to decide which states we use to en- 
code the different possible outcomes of Y. The idea 
is to say that the left well encodes one of the out- 
comes, say "head" and the right well the other. In 
other words, we want the state ph = 1 to encode 
"head" and the state pt = to encode "tail" . How- 
ever, this supposes that we are able to create an infi- 
nite energy difference between the two wells. In the 
following, we assume that there is a maximum value 
Emax > 0, such that \E\ < E'max- Then the best we 
can do is drive the memory to Pmax = Pcq(^'max) or 

1-Pmax = Poq(-£'max)- HcnCC, WC wiU USC Ph = Pmax 

to encode "head" and pt = 1— Pmax to encode "tail" . 
Introducing = Pph + {^ — P)pt7 the marginal prob- 
ability for the particle to occupy the left well at the 
end of the recording process, we can compute the 
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Y = "head" 



Y = "head" 
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1/2 
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Y = "tail" 



Y = "tail" 
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Figure 4: Schematic picture of the recording process 
in the two possible scenarios. The control parameter 
E is driven to E'max or — i?max depending on the out- 
come of Y. Once equilibrium is reached, E is quickly 
driven back to 0. 

maximum amount of information that can be stored 
in this memory using eq. ()13p : 

/max = PD{ph\\p^) + (1 - P)D{pt\\p,n), (22) 

where D{p\\q) = plog^ + (1 — p) log is the 
KuUback-Leibler divergence between two distribu- 
tions over a binary random variable. Figure [3] (a) 
shows Imnx/H as a function of Emax/kBT. For E'max 
of the order of k^T, the maximum information con- 
tent is clearly smaller than H. For -Emax ^ ^bT, 

/max — //• 

Now that we have the states encoding the different 
possible outcomes, we have to specify the standard 
state. Traditionally in the literature, the standard 
state is the state "the particle is in the left (or right) 
well" [31[Tni[5]. However, we can use any distribution 
over the two wells as the standard state. A simple 
choice is the equidistribution: po = 1/2. 

The recording process is schematically sketched on 
figure m If "head" appeared, the parameter E is 
driven from to -Emax and the memory is let to relax 
towards equilibrium ph = Pmax and if "tail" appeared 
we drive E to — Emax and let the memory relax to- 
wards pt — 1 — Pmax- At the end of the recording 
process, we instantaneously drive E back to zero, so 
that it has the same value in the two cases. As can be 
seen on figure |3] (b), at the end of the recording pro- 



/ /max / ^ /max / 

Figure 5: Schematic picture of the erasure process. 
The memory, initially out of equilibrium is simply 
let to equilibrate. The information is erased as the 
overlap between Phit) and pt{t) increase. 

cess, the Kullback-Leibler divergence to the equilib- 
rium state is greater than the maximum information 
stored. 

The erasure process is very simple: We keep E = 
and simply let the memory relax towards equilibrium. 
The erasure process is schematically sketched on fig- 
ure O If "head" was recorded, the initial state of the 
memory is Ph(0) = Pmax and if "tail" was recorded, 
the initial state of the memory is pt{0) = 1 — Pmax- 
The initial information contained in the memory is 
/(O) = /max- As time goes on, ph{t) and pt{t) con- 
verge towards equilibrium, which is also the standard 
state pcq = Po = 1/2. During this process, the infor- 
mation decreases and reaches in the limit t ^ t, 
where Ph{t) = pt{t) = po, see figure El The rate of 
entropy production S'^" jk^ (in units of fce) is indeed 
greater than the rate of information erasure — / at all 
times. Figure [5] (d) shows the time evolution of the 
conditional probabilities P(h|L;t) = Pph{t)/p^{t) 
that "head" was stored observing the particle in the 
left wenandP(t|R;t) = {I- P){l-pt{t)) / {l-p^{t)) 
that "tail" was stored observing the particle in the 
right well at time t. At the beginning of the erasure 
process, they are close to 1. During the erasure pro- 
cess, they decrease respectively towards P and l — P, 
the a priori probabilities that "head" and "tail" were 
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stored respectively. As P(h|L;t) and P(t|R;t) de- 
crease, it becomes more and more difficult to know 
whether "head" or "tail" was stored upon knowing in 
which well the particle is: The position of the parti- 
cle gradually looses the information about the symbol 
that was initially stored. 



6 Conclusion 






(b) 







t/r 




t/i 



Figure 6: Time evolution of various quantities during 
the erasure process for pmax = 0.95 and P = 0.8: (a) 
Evolution of the state of the memory during the era- 
sure process, (c) Information content and total en- 
tropy produced during the erasure process, (d) Con- 
ditional probability that "head" ( "tail" ) was recorded 
observing the particle in the left (right) well at time 
t of the erasure process. 



We have presented the thermodynamics of record- 
ing and erasing information on a physical memory 
within the framework of stochastic thermodynamics. 
Recording some information means to correlate the 
memory to the external source and the information 
contained in the memory is quantified by the mu- 
tual information between the symbol recorded and 
the micro-state of the memory. On the one hand, in 
order to store some information, the memory must be 
out of equilibrium and its average distance to equi- 
librium (measured in terms of Kullback-Leibler diver- 
gence) must be greater than the information stored. 
On the other hand, when the information is erased, 
entropy is produced at a rate greater than the infor- 
mation erasure rate (up to a factor /cb ) . This result is 
a differential generalization of Landauer's principle, 
precisely stating the thermodynamic costs of erasing 
information without making reference to manipula- 
tions on one bit memories. 
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